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ABSTRACT 

Current evaluation activities in the New Jersey 
school system are surveyed, and recommenJations for future evaluation 
efforts are made. The current activities and future developments of 
school (or school district), statewide, and project (or program) 
evaluation are diriscussed individually. The following program 
objectives are suggested: to raise the number of trained education 
evaluators, tc strengthen evaluation capability at each 
administrative level, to install satisfactory evaluation instruments, 
to administer instruments which evaluate pupil performance in 
relation to local and State goals, to expand district evaluation so 
that all schools are evaluated on a cyclical basis, to evaluate 
school districts for Bateman incentive aid qualification, to expand 
evaluation so that all projects financed through the Department are 
evaluated on a cyclical basis; and to increase expenditures for 
evaluation activities. Strengths and weaknesses of several program 
alternatives are discussed. Recommendations for short-range action 
are made concerning school district, statewide, and project 
evaluation, and recommendations for long-range action are made 
concerning needs assessment, management information system, cost 
analysis, instruction, and department task analysis. Additional 
questions for consideration are listed. An analysis of costs and an 
overview of an ETS survey of S.ate testing programs are presented in 
appendices. (KM) 
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I. INTRODUCTION 



In September 1969, the State Board of Education declared Its Intention 
to undertake a thorough examination of current evaluation activities In the 
New Jersey public school system. 

To accompllsFi this aim, a Board Sub-Committee on Evaluation and Testing 
was appointed. Six Board members were named to the Sub- Committee : 

Mrs. Leonard L. Mancuso, Chairman 

Mrs. Hugh Auchlncloss 

Mrs. Marion G. Epstein 

Mr. Calvin J. Kurd 

Mr. George P. Smith 

Mr. V/illlam A. Sutherland 

Ihe work of the Sub-Committee would Include a review of pertinent 
doctjnents and present practices, plus an exploration of various evaluation 
alternatives presented by guest authorities. 

The first meeting of the Sub-Committee was scheduled for October 16. 
A. Proceedings: October 16, 1969 

The chairman opened the meeting by reviewing the charge to the Sub- 
Committee, which was to make recommendations to the State Board concerning a 
Board evaluation philosophy, a Board evaluation policy, and suggested State 
evaluation procedures. The Sub-Committee was asked to recommend the direction 
evaluation activities should take, as well as priorities among evaluation needs. 

Stanley Salett, Assistant Commissioner, Division of Research, Planning 
and Evaluation, discussed three principal evaluation categories and summarized 
the Department's activities in each category: 

1. Project Evaluation - Mandated evaluations of Federally funded 
programs have sharply increased our activity in this area. 

2. School District Evaluation - T!he school evaluation efforts of 
the Division of Curriculum and Instruction serve as a foundation 
for this category. Should the Bateman Plan become a reality, 
school district evaluation would assume major importance. 

3. State-wide Evaluation and Testing - Thus far, the Department's 
activity in this area has been relatively modest, although we 
have kept abreast of promising developments in other states. 
Our proposed Management Information System, however, will give 
us a much greater capability to deal with data analysis. In 
addition, the mandate for a state-wide Title III needs assessment 
program could be the forerunner of a broader state-wide evaluation. 

Since the Department has inadequate resources to undertake a maximum 
effort in all three categories, priorities must be clearly enunciated. 



Following this presentation, meml^ers of the Department's Coordinating 
Council for Evaluation were invited to describe the evaluation activities of 
their divisions. 

Upon completion of these reports, the Chairman asked the Sub-Committee 
to suggest future agenda items. Several members recommended that state-wide 
evaluation be explored at the next meeting. 

B. Proceedings: November 12, 1969 

After reviewing the proceedings of the first meeting, the Chairman 
invited Commissioner Marburger to cbmment on the task facing the Sub-Committee. 
The Commissioner responded by noting the pressing need for both short-term and 
long-term plans for evaluating the issues, problems and concerns of education 
in the State. He also pointed out the necessity of an evaluation of the 
Department of Education vdiich would go beyond that rendered for Title V early 
in 1969. In addition, he st ressed the requirement of an adequate data base as 
the first priority for an evaluation program. To acquire this data base, the 
Department must develop an effective management information system. 

John Casey, Assistant Director, Office of Research, then presented 
the Metropolitan School Study Council's findings on "indicators of Quality". 
This concept bases its evaluation of a school's quality on observable charac- 
teristics of classroom teaching. 

In the afternoon session. Dr. Henry Dyev, Vice President, Educational 
Testing Service, discussed state assessment projects with i*iich he had been 
associated (New York, Pennsylvania, Michigan). After questioning Dr. Dyer 
about various facets of these projects, the Sub-Committee moved to invite 
representatives from Michigan and Pennsylvania to its next meeting. These 
guests would be requested to discuss problems encountered in the initial 
stages of an assessment project, plus the costs involved. 

C. Proceedings: December 16, 1969 

After acceptance of the minutes, the Cl'iaiman asked Prank Pinkowski, 
Assistant Director, Office of Evaluation, to introduce each of the participants 
on the program. 

John Kennedy, Bureau of Curriculum, Pennsylvania Department of Public 
Instruction, was the first speaker. Mr. Kennedy discusi^,ed the Community Needs 
Assessment Project idiich was initiated in Pennsylvania during the current school 
year. 

Fifteen school districts, representing urban, suburban and rural areas, 
had volunteered to take part in the project. Its purpose was to design, in 
each community, an educational program which is based on the community's needs 
as perceived by school staffs, students and parents. 

The Department of Public Instruction provided a liaison staff person 
with technical ability in planning to each district. Ihe initial efforts of 
this planner were centered on staff in-service training and the development of 
lines of communication between the various individuals involved. 



Since the project is less than four months old, it is too early to 
assess its results. In the spring of 1970, however, each district's activities 
will be reviewed, and the next steps will then be determined. It is anticipated 
that districts will have progressed at different rates, so future activities 
will be scheduled on an individual basis. 

Dr. Paul Campbell, Director of Quality Assessment, Pennsylvania 
Department of Public Instruction, was the next speaker^ Dr. Campbell pointed 
out that, by law, his department is responsible for assessing the quality of 
education in the State. To carry out this obligation, it was necessary to 
determine ;diat was to be assessed.; therefore, between 1963 and 1965, a series 
of regional conferences was held to determine the goals of the educational 
system. Ihis effort resulted in a statement of ten education goals for the 
state. 

Dr. Campbell's office is responsible lOr designing the instnaments 
which measure the achievement of these goals. The initial testing effort, 
involving 100 districts, took place during the 1967-68 school year. After 
analyzing the resulting data, refinements of the instnaments have been made. 

Pennsylvania believes that the Quality Assessment; and Community 
Needs Assessment Projects will reinforce each other. A school district might 
use the quality assessment instruments to compare its programs with districts 
having similar characteristics. Its comparative performance might be useful 
in the district's planning operation. 

Dr. Philip Kearney, Associate Superintendent, Bureau of Research, 
Michigan State Department of Education, discussed his state's assessment and 
testing program. In October 1969> the Michigan legislature directed the 
Department to begin to plan and develop a comprehensive program for needs 
assessment and to assess certain Vasic skills during the 1969-7O school year. 

In January 1970, all fourth and seventh grade pupils in Michigan will 
be tested in the area of basic skills. Educational Testing Service has developed 
a test battery under contract with the State. The cost of the battery, -score 
sheets and test scoring is $ .46 per pupil. 

Concurrently, a fifteen-member task force, charged with the responsi- 
bility of drafting a statement of Michig£?.n's education goals, has been organized. 
After the statement has been prepared, a series of regional public meetings will 
be held. Citizen opinion, as voiced at these meetings, will be reviewed by the 
task force prior to its formulation of the final goals statement. The Department 
of Education will then be responsible for establishing procedures for assessing 
these goals. 

Following Dr. Kearney's presentation, the Sub-Committee asked a number 
of questions concerning Michigan's philosophy of testing, activities \A\lch 
preceded the legislative mandate, technical points on the instruments, and 
administration of the program. 

Mr. Robert Locke, Chaiman of the Title III State Advisory Council, 
examined the relationship between a Title III Needs Assessment survey and the 
Sub-Committee's work on evaluation. He pointed out that the Federal Government 



requires that the allocation of Title III funds be based on an assessment of 
the State's education r.eeds. Though such an assessment is presently being 
developed, there are unanswered questions concerning the scope of the project. 

Should thio assessment of needs, asked Mr. Locke, be limited to the 
narrow confines of Title IJI? Or should its scope be broadened to cover the 
full range of the State's education needs? Furthermore, if the latter course 
^s chosen, shouldn't the State Board and the Commissioner assiome responsibility 
for the project, rather than leaving it under the auspices of the Title III 
Advisory Council? 

Dr. Bernard A. Kaplan, Director, Office of Planning, offered a second 
presentation on needs assessment. He asked that the State Board approve a needs 
assessment project and that a special advisory council, composed of representa- 
tives from the State Board, the Title III Advisory Council and lay people, be 
established to guide the activity. The Sub-Committee requested that Dr. Kaplan 
prepare a statement for the Board's approval; he was asked to present it at the 
next meeting of the Sub-Committee. 

D. Proceedings: January 13> 1970 

Following acceptance of the minutes, the Chairman invited Mr. Pinkowski 
to deliver nis presentation on the Belmont Project. (See Part V-B-1 of this 
eport for a discussion of Belmont. ) 

At the conclusion of the presentation, Mrs. Mancuso asked whether the 
Belmont Project would provide adequate data or a suitable conceptual framework 
for a state-wide evaluation system. Mr. Pinkowski replied that Belmont's 
intention was to evaluate the effect of Federal programs and to enable the 
State to develop a coordinated plan for such programs. Belmont will also 
provide infomation for simulation models lAiich can test various formulas and 
strategies for the expenditure of Federal funds. However, Belmont does not 
supply data relating to the total educational program being conducted in a 
local school district. 

Mr. Pinkowski then reported on the National Assessment of Education 
Project. The Sub-Committee recommended that the project director be contacted 
to explore the possibility of using National Assessment materials on a state- 
wide basis. 

After the chaiman reviewed the Needs Assessment discussion of the 
last meeting, Mr. Salett and Dr. Kaplan distributed a statement prepared as a 
recommended resolution for action by the State Board of Education. The Sub- 
Committee asked for. clarification of several potentially ambiguous tems 
("educational goals", "educational objectives", "needs assessment"). It 
suggested certain minor modifications and requested that a revised statement 
be submitted to the State Board on January 26. 

The Chairman pointed out that the Sub-Committee would be asking the 
State Board to approve the determination of broad goals and objectives for 
education in New Jersey. She posed these questions; "is the Sub-Committee 
convinced that it feels the initial effort of Title III should be extended 
into a state-wide goals and objectives program? If so, is this an appropriate 



time to make that resolution to the State Board?" The Sub-Committee unanimously 
answered both questions in the affirmative, and it also agreed to recommend that 
the State Board appoint a special advisory council for the Needs Assessment 
Project. 

The Chairman then asked the group to consider possible future activities 
of the Sub-Committee. Mr. Salett suggested that the Division of Research, 
Planning and Evaluation could prepare a background paper vdiich would siimmarize 
the feasibility and costs of various types of evaluation (school district, state- 
wide, project). The Sub- Committee approved this idea and decided to schedule its 
next meeting upon completion of the paper. 

(Note: The Sub-Committee met on April l6 and May 1, 1970 to discuss preliminary 
drafts of this report. ) 



II, DEFINITION OF THE PROBLEM 



This report accepts the premise that evaluation can have a positive 
effect upon the education of students in New Jersey. It directs its attention 
to current evaluation activities in the State, pointing out unmet needs and 
suggesting ways of satisfying them. 

A. Basic Que_stions 

The nature of a particular evaluation plan can be determined by 
examining it in relation to four basic questions: 

1. What is being evaluated ? Are we looking at one school, at 
a school district, at a project or program, or at students? 

2. What method is employed ? Is it self-study, observation, 
testing, data collection and interpretation, or a combina- 
tion of methods? 

3. Wiio does the evaluation ? Is it conducted by the Department 
of Education, by the local school district, or by an out- 
side concern, or in seme combination^ 

4. Why is the evaluation being done ? For example, do the find- 
ings of the evaluators affect the size--or the continuation- -of 
financial aid? Do we intend to employ evaluation results in 
planning program Improvements? 

B. Emphasis and Impact 

In educational evaluation, emphasis has traditionally been placed 
upon measurement of inputs: class size, expenditure per pupil, age of school 
buildings or textbooks, size of library, teacher preparation. This approach 
overlooks the frequent disparity between the amount of inputs invested by the 
educational system and the quantity that is actually received by the child. 
But more important, there is no necessary correlation between amount of inputs 
and level of student progress. For that reason, it is necessary to focus 
attention on output--the specific changes in pupil achievement which stem 
from a given activity. 

Yet, the latter approach, while logically superior, faces major 
obstacles of its own. For though it might be relatively easy to measure 
academic achievement, how do we assess such intangible goals as the develop- 
ment of creativity, initiative, self-concept, and self-respect? All are 
elusive; they are hard to define and no less difficult to assess. 

Let us assume, however, chat such problems can be overcome. After 
we have devised acceptable indicators and valid instruments, we then must decide 
what to do with the mass of data produced by our evaluations. Hopefully, our 
findings will help us to strengthen current programs and to develop new ones 
which more adequately meet the needs of students. But evaluation should also 
contribute answers to questions of accountability, equal opportunity, distri- 
bution of resources according to need, and significant structural change. 
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Nmerlcal Magnitude 

Ihe sheer size of education in New Jersey is a major factor affecting 
the choice of specific evaluative modes. As of September 1969^ the Staters 572 
operating school districts contained 2,385 public schools, classified as follows 

Elementary Schools 
Junior High Schools 
High Schools 

Vocational or Trade High Schools 
Special Schools for the Handicapped 

Pupil enrollment totalled 1,454,378 (elementary—961, 372 ; secondary— 
493,006), and there were 80,113 full-time teachers and other certificated 
personnel. 

Since roughly 80 per cent of the schools and almost two-thirds of the 
pupils were at the elementary level, they were subject to no uniform evaluation 
program. 

New Jersey's private and parochial schools numbered 660 in 1968-69, 
and they enrolled 312,609 pupils. With few exceptions, these schools \tere 
exempt from mandatory state evaluations. 

D. Statutory Basis 

Evaluation is not specifically mandated in New Jersey's Education 
Law. Several statutes, however, empower the State Board of Education and the 
Commissioner to take action which in effect, if not in name, is clearly 
evaluative. Title 18A:4-10 vests general supervision and control of public 
education (except higher education) in the State Board; it orders the Board 
to "formulate plans and make recommendations for the unified, continuous and 
efficient development of public education. " With the approval of the Board, 
the Commissioner is permitted through a second statute to "prescribe minimum 
courses of study for the public schools and require boards of education to^ 
submit to him for approval or disapproval courses of study adopted by them'* 
(18A:4-25). 

A third statute, i8A:4-24, provides broad authority for evaluative 
activities: "The Commissioner shall, by direction or with the approval of 
the State Board, idienever it is deemed to be advisable so to do, inquire 
into and ascertain the thoroughness and efficiency of operation of any of 
the schools of the public school system of the state and of any grades 
therein by such means, tests and examinations as to him seem proper. " 

These statutes, plus the need of its divisions to carry out various 
State and Federal mandates, undergird the Department of Education's evaluative 
role. Of most importance, perhaps, has been the impetus to rigorous evaluation 
provided by ESEA Title I, Section 205(a) (5), which called for annual evaluation 
of programs and projects. This requirement, subser aently extended to six other 
ESEA Titles, created a climate in which evaluation of activities, with emphasis 
placed upon output, became caranonplace. In New Jersey, the Bateman School Aid 
Plan, idiatever its final shape, should serve as an added prod to strengthened 
evaluation in the schools. 
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E. Need for Action 

Without denying the need fur careful consideration of distinct 
approaches, it should be emphasized that the time for broadening evaluation 
activities seems long overdue. Prom all r.ides, our schools are being pressed 
to offer evidence of their accopiplishments as the price for the public »s 
continuing support. Disgruntled taxpayers have joined critics from outside 
the educational establishment in calling for a thorough accounting of what 
takes place inside the classroom. Both city and suburban parentj, whether 
enraged or merely troubled, are less apt to be reassured by a principal's 
soothing words than formerly. And increasingly restive students, newly 
aware of their rights and power, pose embarrassing questions about the 
validity and utility of the entire school experience. 

Evaluation should help to answer these questions. It should reveal 
more precisely what schools are doing; then it should show how school activities 
affect--for better or worse— the lives of children and their subsequent adult 
performance. 

With this backdrop, we will proceed to examine each of the three 
principal categories of education evaluation. 
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III. SCHOOL (OR SCHOOL DISTRICT) EVALUATION 



Schools are evaluated according to their physical and human resources 
(plant and people--both professionals and students), the nature of their 
curricula, and the breadth of services offered. The schools may either be 
looked at discretely--with each school regarded as an independent entity-- 
or they may be viewed as parts of one school distric"- ^ distinction 
defines the difference between "school evaluation" . 31. district 
evaluation". It is school evaluation which, at the secondary level, is 
now conducted in New Jersey. Recently, however, the Department completed 
a comprehensive evaluation of the Camden School District. 

The criteria by which schools and school districts are evaluated-- 
and approved— may either be established locally (self-study, school plan, 
district plan), or fixed externally (by the Department, some other State 
body, or the Federal Government). The agency which sets the criteria will 
generally determine the relative weight to be given to objective assessment, 
as against subjective appraisal. 

A. Current Activities 

1* The Division of Curriculum and Instruction evaluates and 
approves secondary schools. Each secondary school is 
evaluated every fifth year. The procedure, which combines 
self-study and team visitations, is analogous to that employed 
by the Middle States Association for its decennial accrediting 
evaluations. The Division anticipates extending its evaluations 
to elementary schools, with follow-up guidance provided by 
"educational improvement attack forces". 

Secondary School Approval Procedure 

Title l8i\:45-l and the Rules and Regulations of the State Board of 
Education provide for the approval of secondary schools. Using the Self-Study 
Procedure, approximately 100 public secondary schools are visited each year. 
The Division of Curriculum and Instruction works in close cooperation with the 
county superintendents in these periodic visitations; also, other offices, 
divisions and personnel of the Department are enlisted in the activity. 

An outline of the Self-Study procedure is as follows: 

a. Establishment of annual list of secondary schools 
to be visited and approved. 

b. Numerous county meetings, visits to local school?, 
and office conferences to orient administrators and 
teachers to sel. -study procedures. 

c. DisT^ribution of "Guidelines for Self-Study Procedures 
for State Approval. " 

d. Establishment of Central Office ?J.e., Department of 
Education) team memberships. Assignment of team 
leaders to coordinate and orient team members. 
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e. Procedures developed by local school to study and evaluate 
its program, Participants include administrators, teachers, 
special personnel, students and parents. They will identify 
program strengths, needs and plans >*iich will be published 
in the Self-Study Report. 

f. Visitation made to the school, after review of Self-Study 
Report, by visitation team. 

g. Individual visiting team reports collated and fused into 
a report to the school, which is submitted to the Deputy 
Assistant Commissioner for review and recommendation for 
approval to the Commissioner. 

h. Advisory services and follow-up services provided to local 
school to overcome identified needs and provide assistance 
in developing future plans. 

The typical visitation costs^ for the Self- Study Procedure for a 
secondary school are: 



State Coordination $109.00 

Secretarial Coordination ^6.q6 

Visitation Team; 

a. Central Office Professional (3) 56O.28 
Central Office Secretarial 42.04 

b. County Superintendent & Staff (3) 582.49 
County Superintendent Secretarial 56. ■''5 

State Review and Approval ; 

a. Professional 57*62 

b. Secretarial 17- C4 

Advisory Services: 

a. Professional (2) 18O.6O 

b. Secretarial 11.53 

Total State & County Costs Per School:^ $1,672.72 



During 1969-70, 108 secondary schools--with 114,779 pupils and 6,752 
teachers— will participate in the Self-Study Program. 



Consultations 

The direct contact made by representatives of the Division's Office 
of Secondary Education with individual schools accounts for a great amount of 
the time of each member of the Office. These school contacts are usually at 
the site of the particular school, but often are made in office conferences 
held at the State Department of Education building in Trenton. 
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^ See Appendix A for method of calculating Department of Education salary costs. 
2 Figures based on a normal work day and do not include extended work day, evenings 
and overtime efforts. 
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Telephone calls and written correspondence with people from schools 
are a major form of daily activity, but no effort is made here to eniimerate or 
evaluate the effectiveness of these. They do utilize many man-hours of 
professional time, however. 

The school consultation activity focuses upon matters dealing with 
secondary education in general, as well as on particular curriculim areas, 
such as science and music. Although the primary effort is toward secondary 
education, much of the subject area consultation is with elementary schools, 
especially if they are part of a K-12 curriculum revision program. These 
school contacts, in addition to the school approvals which are scheduled by 
the Office of Secondary Education in cooperation with the County Superintendents, 
are scheduled through a variety of channels. Ihese channels include requests 
made directly to the consultants, arrangements growing out of reviews of 
applications for the approval of course additions, and needs identified in 
relation to the school approval Self-Study, 

A survey was made of school consultations encompassing a period 
of almost two school years (September 1966 to May 19, 1968). The major findings 
of this evaluation were : 

a. Contact with individual secondary schools in the form of 
visitations or office conferences for the purpose of 
providing consulting services were made with 79 P^r cent 

of New Jersey public secondary schools. (Total niomber - 408) 

b. x5y category and number of schools, the visits or office 
conferences for that period were as follows: 

school Approval - 186 
Administration - 75 
CurriculiM, General - Ik 

Pupil Personnel Services - 90 (NDEA, Title V-A) 
ESEA (Titles I & III ) - 11 
NDEA - Administration - IO5 
Art - 6 

English - 35 (NDEi, Title III) 

Foreign Language - 60 (NDEA, Title III) 

Humanities - I8 (ESEA, Title V) 

Math - 2k (NDEA, Title III) 

Music - kl 

Science - 35 (NDEA, Title III) 

Social Studies - 100 (NDEA, Title III) 

TESOL - 10 

c. Schools in all 21 counties received consultant services 
and visitations, 

d. Of the ten largest urban school districts, all were visited 

or hsd office conferences with representatives of the Secondary 
Office, A total of kk separate school contacts were made. 

e. Representatives worked with 299 different elementary schools 
in all but three counties during that time, 

f. Consultant service was provided in all areas of the curriculum, 
as well as in secondary administration, curriculum, guidance 
and co-curricular activities, 

g. Fifty-nine private school visitations were made; forty-three 
of these were in connection with the school approval process. 
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Elementary School Self-otudy Procedures 

During 1969-7O, an elementary school in each of the twenty-one 
ccnnties will pilot-study a "self- Improvement Instrument" for elementary 
schools. The purposes of these twenty-one pilot studies will be to evalu- 
ate the effectiveness of the instrument, ability of elementary staffs to 
engage in self-study, costs and availability of resources, and need for 
in- service pro^/rams for elementary school administrax;ors. The eventual 
goal is to develop a N-12 self-study procedure that will embrace early 
childhood, kindergarten, elementary, middle, jimior and high school levels, 
with a district-wide approach to evaluation and self- improvement. 

At this time, it would be difficult to estimate costs per punil 
to SEA, County Superintendent, and LEA, when the factors under study (: isted 
above) have not been completed. 

Follow- Up Attack Forces 

The Division of Curriculum and Instruction envisions the creation 
of "follow-up attack forces" which will seek to remedy deficiencies foimd 
during the self- study process. Pull germination of ideas concerning the 
follow-up attack forces has not yet taken place. It must, of necessity, wait 
for certain basic decisions, monetary and policy, to be made concerning the 
way in which school approval procedures will expand. 

A tentative projection of costs has been offered ^y Mr. Robert 
Seitzer, Assistant Commissioner. He has estimated that out of a potential 
of 180 days, the attack forces might work I50 days. With an average daily 
force of 10 people, this would be 1,500 man-days. At a daily rate of $50, 
this would cost $75,000. Since these people are to be drawn from local school 
districts, pay for substitute teachers, idiich might average $20 a dav— or a 
total of $30, O00--would be required. 

Total projected annual cost for follow-up attack forces: $75*000 
plus $30,000 - $105,000. 

Other school evaluation activities undertaken by the Division of 
Curriculum and Instruction include the inspection of more than 60O non-public 
child-care centers; the approval of secondary school courses and summer sessions; 
and special evaluations of schools and districts upon the request of the 
Commissioner and the State Board of Education. These activities are mandated 
by Title i8a or State Board regulations. 

The Division of Business and Finance audits the financial records of 
all public school districts; holds budget hearings when local school budgets 
are defeated by the electorate; approves extension of credit when legal debt 
limits must be exceeded for school construction; and prepares the Commissioner's 
Annual Report of Comparative Statistics. In addition, the Division's Bureau 
of School Building Services reviews all specifications and blueprints prior to 
facility construction; and when new facility requirements are instituted, it 
checks for compliance. 

The Division of Vocational Education registers private vocational 
schools, licenses their teachers, and conducts periodic evaluations. The 
following outline summarizes these activities: 
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A* Registration and Regulation of Private Schools 

1. Legal basis: N.J.S. i8A:69-1 through l6 

a. "standards for the Regulation of Private Trade 
Schools" (printed). 

b. "Checklist for Establishing Private Vocational 
Schools " (printed ). 

2. Procedures for Application and Approval 

a. Letter of request to establish a private trade 
and technical school from applicant. 

b. Interview of the applicant. 

c. Inspection of facilities. 

d. Review of plans for administration and instruction; 
assisting in development of such plans. 

e. Comprehensive inspection of a?! items pertaining 
to total school operation. 

f. ^ Issuance of appj^oval certificate. 

3. The school's certificate so approved is valid for only 
one year. Renewal of the certificate should proceed 
as follows: 

a. Ti'aining establishment submits renewal data on 
fom provided. 

b. Renewal application evaluated. 

c. Inspection visit of physical facilities to insure 
total compliance with rules and regulations. 

d. Issuance of approval for following 12-month period. 

B. Teacher Licensing 

1. Legal Basis 

2. Procedures 

a. Submit written application and certain documents. 

b. Evaluation of the application. 

c. Issuance of appropriate license in compliance 
with rules and regulations. 

C. Evaluation of Private Trade Schools through Visitation and 
Review of Annual Report for Fulfillment .of Request for 
Renewal Approval 

1. Legal Basis 

2. Procedures 

a. Visits are made by professional staff of the 
Private Trades and Technical Schools Office. 

b. A minimum of three visits per year are anticipated, 
more if problems exist. 

c. Visitation reports are filed in the school's 
folder for future reference. 
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The annual cost of evaluating each of the 6l approved private 
trade and technical schools is $211, This approximate figure breaks down 
into the following components; 

1. Manpower? $176.00 
4 man- days (professional) per school, 
consisting of 3 visitation days, plus 

one day of office work 

One day (secretarial) 18.00 

2. Travel 12. 00 

3. Miscellaneous 5« 00 

4. Total $211.00 
6l schools X $211.00 - $L2,871 



The Camden Report (properly titled A Survey of the Camden City 
Public Schools ), published in November 19^9, was the Department's first attempt 
at a comprehensive evaluation of an entire school system. All aspects of the 
district's educational program were examined. The district's strengths and 
weaknesses were pointed out, and copious recommendations were offered. Individ- 
ual reports were prepared on the following topics: 



Elementary School Program 

Secondary School Programs 

Special Services 

Health Services 

Vocational Education 

Educational Facilities 

Pupil Transportation 

Revenues and Expenditures 

Accounting and Business Practices 

Program Audits: School Lunch/Milk ^ MDTA; Title VI-A 

School Library Services 

Equal Educational Opportunity 

Racial Balance 

ESEA Title I and III Programs 



In addition, the Camden survey carefully explored community. State 
and National factors i*iich have an Impact on the quality of education in the 
city. The survey took slightly more than one year to complete, because most 
staff were participating in it while fulfilling their regular assignments. 



^Salary expenses shown in each cost breakdown are supplied by the division 
responsible for the activity. There may be some variation in average pro- 
fessional costs for different activities. 
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The survey enlisted 'che aid of ll8 Department professionals, plus the 
services of outside consultants. Here is a brief summary of their activities: 

a. Interviews were conducted with the chief administrators 
and members of the Board of Education to docioment their 
perceptions of the most severe problems being confronted 
by Camden. 

b. The Office of Federal Assistance conducted a careful analysis 
of the Title I program, including visits to Title I schools, 

a review of the Title I application and an attempt to estab- 
lish the role of the community in the Title T program. 

c. The Division of Business and Finance conducted a spot 
audit of the books of the Camden School System, visited 
each school building to determine its safety and condition, 
and reviewed the financial condition of the city. School 
transportation procedures were reviewed. 

d. Tfeams from the Department were established to visit every 
sch'jo] In CT^c'en, and to take note of the atmosphere and 
activ lu: 3^ -^ich school. This included an analysis of 
librurj r-er j, audio visual resource^, the attitudes 
of teacxiei^ ana principals, and unusual programs. 

e. The Divir"*on ox Vocational Education conducted a two-day 
review o-. vocational programs and facilities. 

f. The Office of Equal Educational Opportunity reviewed the 
problams of segregation and integration, plus the progress 
being made in developing curricula relating to blacks and 
Puerto Ricans. 

g. All available statistics and information were provided by 
the Department's Office of Statistical Services for review 
prior to on-site visits and for use in the preparation of 
this report. 

h. Outside consultants were employed by the Department to 
assist in the review and to supplement reports in various 
areas (desegregation, ESEA Title III, school system 
organization and management, overall appraisal). 

In all, the Camden School system evaluation comprised 426 man-days of 
on-site visits by the ll8 Department professionals, plus half of that amount of 
time devoted to report writing. An additional 50 days was contributed by ten 
outside educational and editorial consultants. No record was kept of time 
spent by LEA and community people in Camden or by Department non- professional 
employees. 

At an average rate of $80 a day (salary, fringe benefits, travel, 
administrative overhead), the 684 professional man-days represented an 
expenditure of approximately $55,000. Based on that amount, unit cost figures 
for Camden schools and pupils are as follows; 

Average Cost (jSchool ) 



1. Number of schools: 

2. Evaluation costs: 

5. Cost per school (StI) 



32 

$55,000 
$ 1>719 
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Average Cost (Pupil) 



1. Number of pupils- 

2. Evaluation cost : 

3. Cost per pupil: (2fl) 



20,146 (as of September 30, 1968) 
$55,000 
$ 2.73 



It should be emphasized that since the Camden survey was the first time 
the Department had undertaken such a comprehensive evaluation, future efforts of 
this kind might be done more rapidly and economically. 



B. Future Developments 



1. The Bateman Plan -- The report of the State Aid to School Districts 
Study Commission^, whose major recommendations are embodied in a 
bill presently before the New Jersey legislature (Senate 575 ), 
proposes a thorough revamping of the State's program of fiscal 
aid to local school districts. Under the suggested plan, six 
separate district classifications would be recognized. An 
incentive equalization program would be adopted; under the 
new formula, districts will be "granted assistance in relation 
to their educational responsibility and fiscal capacity, with 
higher State support granted to districts which operate better 
quality programs.-^" 

Ihe above citation summarizes three major provisions of the Bateman Plan 

a. The "educational responsibility" of a school district is 
not measured merely by the number of students it enrolls. 
Rather, the proposed plan recognized that it is more 
expensive to educate children from low- income families, 
those in vocational schools, and those in hi^er grades. 
As a consequence, pupil units are weighted according to 
the estimated cost of educating these different groups. 

b. The financial resources of a district, as measured by the 
equalized valuation of its taxable property, will affect 
the size of State support. 

0, Districts i*iich provide "quality" education will be 
rewarded with higher State support than those offering 
minimally acceptable programs. 



The latter provision should constitute a powerful impetus towards 
educational improvement. It also will ensure that school district evaluation 
takes on major importance. 

The Bateman legislation, in its present form, does not prescribe the 
specific qualifications for each district classification. Instead it delegates 
that responsibility to the Commissioner of Education: 

"For the purpose of computing State Aid, the Commissioner, with the 
approval of the State Board, taking into consideration the quality of the 
educational program and the organizational structure of the districts, shall 

A State School Support Program for New Jersey (Bateman Report), 



5 Ibid, p. 39 
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determine criteria and standards to be used in Judging what shall constitute a 
non-operating district, a basic district, a limited district, an intermediate 
district, a comprehensive district and a superior district. Such criteria and 
standards shall be formulated annually. 

If a school district wishes to be classified in a category higher 
than Intermediate, its school board must file a request through a report to 
the Commissioner: 

^'The report, with supporting evidence, shall describe the program 
lAiich will be in effect during the school year for which the calculation of 
State Aid will be made and it shall contain a written statement of the board ^s 
philosophy and objectives of education as well as the boara's p r oposed program 
of evaluating the attainment of the board's philosophy and objectives^" 7 



^ An Act Concerning State Aid to Education. (Senate, 575) P- 5> lines 26-34. 
7 Ibid, p. 6, lines 58-64. Emphasis added. 
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IV. STATE'WIDE EVALUATION 



This category involves the collection and analysis of information 
about the inputs, process, outputs and environment impinging on education 
throu^out the State. Input data concerning school facilities, instructional 
staff, pupil characteristics, and canmunity characteristics and financial 
ability are gathered. Output data relating to potential and achievement are 
gathered to provide information on the educational progress of the state 
system. The product of this evaluation should be enhanced capacity for 
intelligent decision making at both the state and local level. The data 
may be organized in a variety of ways: for the state as a whole, by 
geographical region, by individual school districts, or by school district 
types (urban-suburban-rural, high-medium- low socio-economic level). Both 
attitude surveys and hard data may be utilized. And state-wide pupil 
testing is a potential part of this category. 

In New Jersey, the anticipated installation of a Departmental Manage- 
ment Information System should give us the potential capability for this type 
of statewide evaluation. 

A. Current Activities 

State Departments of Education across the nation are becomin^;:; In- 
creasingly concerned with requirements for some form of accountability In the 
educational program of the schools. The degree of concern Is In direct propor- 
tion to the state's share of the financial support of the education program. 
As legislatures approve funding vftilch accounts for 40 per cent or more of the 
cost, there Is greater concern for better accountability of the expenditures. 

Three distinct forms of effort can be identified as attempts by State 
Departments to develop solutions for accountability. These are categorized as: 

1. Attempts to re-orient local districts toward defining 
their programs in output or operational goal statements. 

2. Attempts to develop sounder and more inmiediate data 
bases for evaluation purposes. 

5. Attempts to identify and deal with the difficult to 
quantify human relationships and processes which are 
the critical factors in the organization's effectiveness. 

There is a wide-range variance between states in these efforts. The 
majority, at the present time, are working simultaneously within the first and 
second categories. Very few have come to grips with the third category. The 
Interest of at least 27 states in the first two categories is evident by their 
participation in the Belmont Project, a systems approach to the evaluation of 
Federal funding in education. 

During the 1967-68 school year. Educational Testing Service conducted 
a national survey of state testing programs. For the survey, a state testing 
program was defined as any organized, coordinated, centralized effort to 
provide some type of test material? or services on a state basis. Employing 
this definition, it was possible to distinguish 7^ testing programs in 42 
states with 18 states offering two or more programs. There was a range from states 



furnishing virtually every conceivable service associated with testing to 
states that merely offer assistance to schools or school districts In develop- 
ing and/or improving their local programs. 

For example, a number of states recommend that one or more tests be 
administered In the public schools as a basis for the authorization of Federal 
funding. These states typically provide a listing of tests from which local 
schools may select those they wish to administer. 

(Note: An overview of the study as presented by Educational Testing 
Service is given as Appendix B to this report). 

State testing practices are varied in every aspect of procedure. Proce- 
dures can be varied in four areas: (1) test construction, (2) administration, 
(3) scoring, and analysis and reporting. 

Test construction is Identified within three areas: 

1. Department or state institutions constructing the tests. 
The Iowa Test of Basic Skills and the Iowa Test of 
Educational Development are constructed in the College 
of Education at the University of Iowa, 

2. Department contracted test construction with testing 
agencies. Michigan and Ohio have contracted with ETS 
to construct testing batteries for their respective 
states. 

3. Department listing commercial testing batteries on 
approved lists. 

Test Administration 

The questions of "who," "what," "when" and "how" are administrative 
decisions. The "who" ranges for students in grades one through twelve. 
California has reading tests which are mandatory in grades one, two, and three. 
The common pattern is to test at the elementary level for basic skills and 
at the secondary level for educational developnent . In California in 1971-72, 
a student must show an eighth-grade performance level to receive a high school 
diploma. 

Tests are administered generally in the early fall -- usually 
October. At the elementary level, grades three and six are most common. 
Grades nine and eleven are most common at the secondary level. 

How tests are administered varies from direct administration by 
testing bureaus within State Departments to University affiliation, as in 
Florida or Iowa. The general pattern is to send the test packets to the school, 
where supervision is given by local personnel. 

Test Scoring 

Test scoring has a variety of patterns, with the most common method 
being local responsibility. The local district can hand-score the test, 
using teachers, or contract with test scoring service bureaus. In this instance, 
results are sent into the Department. 
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Ohio scores its testing battery In the testing bureau. In Iowa, 
a corporation has been set up to do test scoring. Many other states have 
contracted with ETS to score the tests. 

Analysis and Reporting 

Where testing is state-wide, either mandatory or voluntary, the 
common practice for State Departments is to assemble the scores and report 
to the districts on established norms. Norms are usually compared between 
schools within the district, a larger geographical area such as the county 
or the region, and the state. In all instances, the privacy of the district 
is protected by guarantees between the Department and the districts of the 
confidential nature of the information. The use of the scores is a mutual 
agreement contract. 

Financing 

The cost of state testing practices ranges from a low of $ ,46 per 
pupil to $2.00 per pupil. In Michigan, where the legislature appropriated 
$250,000, the Division of Research, Planning and Evaluation contracted with 
Educational Testing Service to test all fourth and eleventh grade students 
in the state in the achievement of basic skills. 

The common pattern across the nation is local district purchase of 
the service; e,g,, Iowa and Ohio, However, there are variations. In 
California, districts are required to budget $1,75 per pupil for the testing 
program. 

In New York, all third and sixth grade pupils are tested in reading 
and arithmetic. The cost of this program is paid from Title I funds. 

In the Quality Assessment Project in Pennsylvania, the cost is 
approximately $1.25 per pupi!-, and is carried by an appropriation from the 
state legislature. This is a limited program using a sample of IOC schools 
and two grades within those schools. 

Evaluation using tasting practices in the difficult areas is underway 
in New York, California, Iowa, Pennsylvania and Michigan, Florida is exploring 
a number of possible approaches, beginning with a five-year phased management 
information system plan. Their investment cost in the equipment and staff is 
$1,5 million at this time. 

B, Future Developments • 

1. Management Information System, The Department* has established 
as a high priority the development of a management information system (MIS). 
The plan is designed as a three-stage project which culminates in an operational 
system at the end of two years. Phase I has been approved and has now begun. 
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The three stages are identified as: 

a. Exploration and Design 

b. Development and Field Testing 

c. Operation 

The major characteristics of a management information system 
are uniqueness and flexibility. The system is unique in that It must be 
designed for particular management requirements. There is no stock model 
or catalogue of existing models from which the Department can make a selection. 
The requirements for a Department model are translated into specifications 
which must be developed. 

Flexibility is the second major characteristic. This is translated 
into an open-ended system with capabilities for expansion. Experience has 
proven that no major operating management information system developed is fool- 
proof, complete design. As systems become operable, new data requirements 
are generated from the user level. It follows that as people learn, they 
have a desire to want to know more. 

The system for the Department needs to be flexible because of the 
major requirements imposed by the Department. The first requirement of tho 
system is to provide a more efficient means through machine processing for 
carrying out Department data routines. 

The second requirement is an analytical capability for processing 
data. The system would provide staff with statistical models to provide 
information for decision-making and/or the opportunity to field-test models 
whiCxx are staff developed. This is a planning, evaluation, and research 
capability. 

The thiixi requirement is a quick-query system for top-level manage- 
ment. The system would have the capability of quick retrieval of information 
upon the request of management for on-going operations. An example of such 
information would be the number of children enrolled in non-graded primary 
classes and their location in the state. 

The Department is currently engaged in the activities under Stage I. 
Through the State Department of Treasury, bid proposals were solicited from 
management consultant firms to develop the specifications for an information 
system. In this study, a thorough analysis would be made of the Department's 
data operations, with documentation and a conceptual design of a system 
presented for approval. The specifications would be written for the operational 
requirements of the design and a development schedule. 

2. Evaluation Models - As an adjunct to its work on a management 
information system, the Department has investigated a variety of evaluation 
systems and has initiated the development of two models. Proposals of evaluation 
models have been submitted to the Department by Dr. Malcolm Provus of the 
Pittsburgh Public Schools and the E. F. Shelley Corporation. 
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The principle of discrepancy underlies the Pr^'^us Model. Discrepancy 
is described as the measured difference between the expected and the actual 
outcomes of a program. The problem for which the solution would be developed 
is the identification of key variables in the measuring process. Simply stated; 
"Can five variables be as valia as twenty- five?" The management information 
system, in this case, wuald be field-testing the model. 

The E. F. Shelley Corporation proposal is essentially directed toward 
school district evalunr.'Dn. It ir called an educational audit. The sum total 
of all district audits the state evaluation. The proposal as presented is 
in its initial stage of development. It proposes to use the available data 
at the school distric and use opinion questionnaires with teachers and 
students to correlate attitude with achievement. As in the Provus proposal, 
field-test/ng plus further design and development are necessary. The development 
costs of tnis system are estimated at $1 million over two years. 

3. National Assessment - The National Assessment of Educational 
Progress project has been in operation slightly less than six years. During 
the 1968 and 1969 school years, the first assessment i: -truments were 
administered to the population. The results are being anb.'yzed and a report 
is due in the spring of 197O. The Project Board is er^ploring the possibiliL^ 
of working with states to develop state evaluation systems based on the instru- 
ments developed by the project. 



V. PROJECT '(or ]hOO?AK) ::v;-LUiuiQ; 

Ti-'-^'s category comprises the evaluation of specific project:., 
eitner in response to Federal mandates under categorical aid prorrar.s^ 
(ESEA Title I, III, etc.) or as required by a Department of r^ducatlon 
directive. In addition, individual school programs (reading, career 
development, sex education, physical education, etc. i are frequently 
selected for special examination and analysis. 

A. Current Activities 

1, The Division of Vocational iducation is charged v;ith the 
evaluation of Federally funded exemplary vocational projects. The 
responsibility of evaluating a particular exemplary project lies v;iti. 
the project director or supervisor.* In order to fulfill this responsibi 
ne has to secure sufficient evidence to rnswer two questions: namely, 
U) To what extent has the project been implemented: {>:) To vhat extent 
has the project achieved its objectives? Following is an outline of the 
major steps for evaluating an exemplary project: 

a. List the major tasks (services or activities) the 
project applicant is committed to perfonrj. 

b. List the objectives of the puject as proposed by the 
project applicant and approved by the Division. 

c. Define the objectives in behavioral terms and devise 
measuring scales for v;eighing surcess or failure for 
each of the objectives. 

d. Develop instruments for collecting factual data with 
respect to both project implernent'ation and achievement. 

e. Collect data. 

f. Process the data for the purpose of answering the two 
central questions: 

1. To what extent has the project been 
implemented? 

2. To what extent has the project achieved 
its objes^cives? 



-For projects of relatively large scale, evaluation committees may be 
organized to serve in an advisory capacity. 
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g. Write the evaluation report, using this outline: 



1. A brief description of the project 

2. Major tasks (services or activities) 
and objectives 

3. Evaluation design in brief 

4. Findings and conclusions 

5. Recommendations 

6. Appendices (measuring scales, 
data collecting instruments) 



h. Submit the evaluation report through the proper channel. 



To evaluate each of the 256 projects currently operating under 
the Pilot Project Program, the following estimated annual cost is anticipated 

1. Manpower 

1 man-day (professional) per project 
J day (secretarial) 

2. Travel 

3. Miscellaneous 

4. Tct-i'l 

256 projects X $68 = $17, 4o8 




2. The Office of Federal Assistance directs evaluation activities 
for ESEA Title I projects and coordinates evaluative procedures for other 
Federal programs. 

Title I grants are made to 430 school districts, with pupil partici- 
pation totaling approximately 68,000. The State's Title I allocation for 
Fiscal Year 197O f^ roughly $18.5 million — $300 per child. Twenty-six 
school districts, accounting for 46, 154 Title I participants, each receive 
more than $100, 000. They are: 



Asbury Park 

Atlant1.c City 

Bayonne 

Bridgeton 

Camden 

East Orange 

Elizabeth 

Franklin Township 

Hackensack 

Hoboken 

Jersey City 

Lakewood 

Long Branch 



Neptune Township 
Newark 

New Brunswick 
Orange 
Passaic 
Paterson 
Perth Amboy 
Pla infield 
Pleasantville 
Toms River 
Trenton 
Union City 
Vine land 
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Title I evaluations are customarily performed by local school dis- 
trict personnel. The LEA coordinators of Federal projects, plus a varying 
number of clerical er.ployees, carry the major responsibility. For cost and 
work unit data, the 1968-69 Hoboken Title I project can serve as an example 
of the resources needed; 



a. Total Title I Allocation $285,9^8 

b. Evaluation Cost 1, 500 
(30 man-days — professional and 

clerical — at an average rate of 
$50 per day) 

c. Percentage of evaluation cost to total 
allocation (a f b) .^2k% 

d. Evaluation cost per Title I pupil $ 1.25 



In some instances, outsioe consultants are hired to evaluate Title I 
projects. In Newark, the 1968-69 project was examined in a year-long evalua- 
tion by Planners Associates Inc. This evaluation, which included in-depth 
studies of sixteen Title I schools, was conducted by 17 personnel (13 pro- 
fessional, k non-professional). 

a. Total Title I Allocation $4,443,504 

b. Evaluation Cost 98,000 
(Contracted price; log of man-days 

worked not available) 

c. Percentage of evaluation cost to 

total allocation (a *■ b) 2.2^ 

d. Evaluation cost per Title I pupil $ 3.45 

The annual evaluation reports from the 430 Title I districts are 
sent to the Office of Federal Assistance, where they are reviewed for statis- 
tical and program accuracy, collated, printed, and sent to the Title I 
coordinators of all other states and to the U. S. Office of Education. 



3. The Division of Research, Planning and Evaluation assists in 
strengthening the evaluation process mandated for Federal ESEA Title III 
projects. Each project is subject to two evaluations r 

^» Local . The local school district arranges an evaluation 
of its own project. LEA personnel may conduct the 
evaluation, or it may be done under contract with an 
outside consulting firm. If a consultant is employed, 
the cost might reach as high as $5>000. New project 
applications must earmark a minimum of 5^ of the grant 
for evaluation. 



b. Central . This evaluation is coordinated through the Division 
of Research, Planning and Evaluation. It may be either an 
Audit Evaluation, involving 1-2 people from within or 
outside the Department, costing up to $300; or It might be 
a Team Evaluation, with charges ramging up to $1, 200. Team 
Evaluations may be accomplished by Divisional staff members, 
by sub-contract with an intermediate unit (Educational 
Improvement Center, Urban Schools Development Council), 
Q or by contract with an outside consultant. 



ERIC 
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Because of the wide variation in project type and scale, it would 
be meaningless to compute an average cost per pupil for Title III evaluations, 

B. Future Developmen ts 

1. Belmont Project , The Belmont Project is a Joint agreement 
between the Chief State School Officers and the United States Office of 
Education to cooperatively develop a consolidated evaluation system for 
federal programs. Thi*^ agreement, the first in the history of education, 
was made at the August 1968 meeting of the Chief State School Officers. 

It was the understanding that in Fiscal Year 1969 a broadly 
representative group of states, mutually agreed upon, would participate in 
pilot instrument development. New Jersey was one -f the original seventeen 
states selected and has participated in all Belmont Project meetings. In 
Fiscal Year 1970, the number of states had expanded to twenty-six. 

The first meeting of state representatives and Office of Education 
staff was held in January 1969 at the Smithsonian Conference Center's Belmont 
House in Maryland. Three committees were organized r 

a. Consolidated Program Information Report Committee 

b. Instrument Design Committee 

c. Staff Training and Dissemination Committee 

Since January 1969, there have been five national and two regional 
meetings. The national meetings have been work sessions, while the regional 
meetings have been instrument review sessions with local school district 
representatives. 

The target date for the implementation of the final system is the 
1972-73 school year. The work breakdown structure includes the development 
of questionnaires, field tests of the questionnaires, analyzing the informa- 
tion, and refining the questionnaires. 

The total system is divided into four sub-sections for which 
information is collected: 

a. Consolidated Program InfoiTnation Report (CPIR) instrument 
is developed and currently being field-tested in eighty- 
se^'en New Jersey districts. This instrument reports all 
Federal programs by source of funds in a school district. 

b. Pupil Centered Instrument (PCI) is a battery of four ques- 
tionnaires (1. school district; 2. building principal; 

3. classroom teacher; 4. student). The purpose of the 
instrument is to describe the environment of the project. 
Intensive work was done on this instrument during the 
past winter. It will be field-tested in April, 197O. 
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c. Project Descriptor Instrument (PDI) is in the opening 
stages of development. It will not be ready for field- 
testing until the fall of 197O. 

d. Common Status Measure (CSM) instrument is in the planning 
stage of development and will not be field-tested before 
the spring of 1971. 

The purpose of the system is to consolidate the reporting and 
evaluation requirements for the following legislation; 

Elementary and Secondary Education Act - Titles I, II, III, 

V-503, VI, VII and VIII 
National Defense Education Act - Titles III and V-A 
Civil Rights Act - Title IV 
Follow Through 

Vocational Education Act of 1968 

2. Section 402, Title IV, ESEA Amendments of I967 , Under this 
legislation, $100,000 will be allocated annually to each state during Fiscal 
Years HO, ^Jl, and '72 for the planning and evaluation of Federal programs 
or projects. These funds may be used for the following purposes r 

*a. Staffing and supporting a planning and evaluation component 
in the State educational agency. 

b. Training SEA staff in planning and evaluation procedures 
and techniques. 

c. Providing consultative and technical assistance and services 
in planning and evaluation to intermediate and local educa- 
tional ag'^ncies. 

d. Designing, developing, installing and ma'lutaining planning 
and evaluation systems and procedures in the State educational 
agency, and conducting appropriate studies and surveys. 
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D. To administer instruments which evaluate pupil performance in 
relation to local school district goals and to each of New Jersey's 
education goals. 

Data Indicators 

1. Number (or %) of State education goals evaluated by adminis- 
tering approved instruments to representative pupil 
populations 

2. Number (or^) of pupils by age (or grade level) evaluated 
against individual State education goals 

E. To expand £3chool district evaluation activities so that all 
schools in the State (Nursery through Grade 12) are evaluated 
on a cyclical basis 

Data Indicators 

1. Number (or %) of all State schools participating in a self- 
study/visitation evaluation program (year) 

2. Number (or %) of all students in the State attending schools 
which participate in a self-study/evaluation program (year) 

3. Number (or %) of all State schools participating on a cyclical 
basis in the established evalixation procedure 

F. To evaluate school districts for Bateman incentive aid qualifi- 
cation ^^^'"^^ 

Data Indicators 

1. Number of school districts (or %) evaluated according to 
Limited requirements (year) 

2. Number of school districts (or /S) evaluated according to 
Intermediate requirements (year) 

3. Niomber of school districts (or %) evaluated according to 
Pre-Comprehensive requirements (year) 

U* Niomber of school districts (or %) evaluated according to 
Comprehensive requirements (year) 

5. Total number (or %) of school districts currently holding 
Limited status 

6. Total number (or %) of school distri^"- currently holding 
Intermediate status 

^^^This objective assumes that Statewide goals will be established in the near 
future. 

•'^•'^'^Specific criteria for districts remain to be established. 

ERLC 
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7. Total nur.ber (or %) of school districts rarrer.tly hcliir:.: 
?re-Cor:prehensive status 

a. Total n-jnber (or of school districts c*-irrer.tly holding 
Cor-prehensive status 

To ex:>and project evaluation activities so that all projects 
firianced through the Department -^11 be evaluated on a cyclical 
oasis • 

Data Indicators 

1. Number (or ?) of all projects evaluated (year) 

2. Number (or %) of all projects participating in a cyclical 
evaluation procedure. 

To increase expenditures for evaluation activities 
Data Indicators 

1. Total dollars spent on evaluation activities throughout 
the State (plus year-to-year % change) 

2. Total dollars spent on evaluation activities as a ^ of all 
education expenditures (through grede 12) in the State 

3. Dollars spent jn standardized test administration (plus 
year-to-year % change) 

I. Dollars spent on instrument development (plus year-to-year 
% change) 

5. Dollars spent on training of evaluators (plus year-to-year 
% change) 



D. 



Dollars spent on analysis of data (plus year-to-year % change) 



7. Dollars spent on information dissemination (plus year-to-year 
% change) 
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VII. PROGRAM ALTERNATIVES 



A. School Approval Model 

The concept of school approval is based on a school building as the 
unit for evaluation. The process is a combinatio.i of self-study by the staff 
and an approving team visitation. 

The cost analysis of the concept, based on the assumption that a school 
would be approved on a five-year basis, is-: 

Number of school buildings in New Jersey = 2,500 + 
Composition of team = 8 for secondary & 5 for elementary 
Period of team visitation = 3 days 
Cost per person on team = $50 per day 
Operation of schools = 36 weeks per year 
Number of schools visited per year = 50 secondary 

kkO elementary 

Number of schools evaluated per week = m- 
Staff requirements for Ik teams = 73 
Total Cost of Program = 

219 man-days per week X 36 weeks X $50 per man-day = 

$39^,200 annually 

Cost does not include administrative overhead. travel, expenses, 
secretarial, printing reports, etc. 



Weaknesses 

llo guarantee of the generation of compatible data to provide state 
management with dec is ion- making information. 

The concept is not compatible with the proposed Bateman legislation 
as it relates to district classification rather than school classification. 

Five years between approvals is a gap liiich makes It impossible 
to establish priorities based on problem identification. The state 
decision-maker would be concerned with the problems identified in 490 
schools, 10 per cent of v^ich are secondary. 

Strengths 

Ihe capability of moving Into a single school as the unit of evaluation. 
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B, School District Evaluation 

School district evaluation can fall Into three categories: 

1. School district approval 

2. School district quality assessment 

3. School district audit model 

To our knowledge, there are no satisfactory operational models of 
school district evaluation systems In existence at the present time. Even If 
there were models, It would be necessary to make modifications In line with 
New Jersey requirements. 

1, School District Approval 

The school approval can be broadened to the district level as the unit 
of evaluation. It would follow the process of self- study plus evaluation team 
visitation. There are no standards for team size or scheduling, but from the 
Camden experience, the size could range from five to twenty- five persons. 

The cost analysis of this donoept is based on the assumptions of a 
five-year cycle of approval and an average team size of twelve persons. 

Number of operating districts in state = 574 

Number of approvals per year = 115 

Average number of persons per team = 12 

Period of team visitation = 5 <?«vs 

Cost per person on -team = $50 per day 

(deration of schools = 36 weeks 

Number of districts evaluated per week = 3.2 

Staff requirements for 3.2 teams = 38,4 
Total Cost of Program = 

192 man-days per week X 36 weeks X $50 per day = $3^5.600 

Cost does not include administrative overhead, nor does it reveal 
hidden costs of follow-up of evaluation. 

Weaknesses 

No guarantee of the generation of compatible data to provide state 
management with decision-making information. 

Five-year gap between approvals is unrealistic in a changing environment 
and is not ccanpatible with Bateman legislation for an annual process. 

Does not provide a data base. 

Strengths 



Maintains the capability of moving into a single school as the unit 
of evaluation. 



33 



Process has the capability to review a coordinated K-12 program. 

Meets the requirement unit of school district stressed in the 
proposed Bateman Legislation. 



2. School District Quality Assessment 

School district quality assessment is a systematic approach to the 
measurement of the goals and objectives of education which are held to be common 
for all school districts. It would require a developmental period of from two to 
four years before an operational program was fully implemented. 

The first step in school district quality assessment is the identifi- 
cation of the goals idiich would be stated as the common object.^ves for all the 
school districts of the state. Having determined the goals, the second task is 
the development of instruments to measure the objectives. This stage has no 
terminal point and becomes a continuous operation of instrmnent design for more 
discrete measures. 

Stage three is a field test operation of the designed instriiments for 
validity and reliability. Stage four is the generation of an educational profile 
from the data collected by the instruments. The educational profile, displaying 
a state norm for each of tho objectives, is a report of the condition of the 
educational program of the state. Any district in the state could take its 
data from the instruments and compare its condition against that of the state 
norms. 

A cost analysis of this concept is more difficult to arrive at in the 
context of this paper. The preceeding system's costs were operational, and for 
comparison purposes, the operational cost of this system should be used. The 
reader should be aware that the system would need to be developed, and development 
costs are investments which, of necessity, are higher than operating costs. 
Therefore, the following assumptions are made for the purpose of comparable 
analysis of cost: 

1. The system is operational 

2. State norms are developed annually 

3. Grades k, 8 and 11 are the base of assessment 

4. A state sample will be drawn of 150,000 students 

5. The cost is $1.50 per student 

The annual cost to the state would be $225,000. This cost would 
include the salaries of a permanent staff assigned to quality assessment with 
the responsibility of developing the annual educational profile. It would cover 
the costs of the school districts drawn in the sample from state funds. The 
remaining districts would bear the expense for developing their norms from local 
funds. 

Weaknesses 



No guarantee of all school districts participating in the assessment. 
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Current lack of valid instruments to measure objectives in the 
affective domain. 

No follow-up procedure other than annual reports. 

Is limited to common goals and objectives of all schools. 

Strengths 

Provides state management with a current picture of the educational 
program and capability to establish priorities. 

Department capability to identify problem areas for further 
investigation. 

Gives school districts the capability of assessing thf-ir programs. 



3. School District Audit Model 

Ihis concept is experimental and has not entered the developmental 
stage. The basic purpose is to develop a system which identifies and analyzes 
educational problems within a school district. The analysis would attempt to 
establish the cause and effect relationship existing in the identified problems 
and make recommendations based on this evidence. 

Ihe concept combines the visitation feature of the district approval 
model with the analysis feature of quality assessment. However, it proposes a 
greater depth to the analysis than is contained m quality assessment. Quality 
assessment identifies the existence of a problem; the audit proposes to determine 
why it is a problem. 

A school district audit requires a team with the ability to identify 
data needs and make the analysis. At this stage, the experimental, the analysts 
are not certain of what data is necessary and propose to conduct studies to 
determine valid and reliable measures. 

A proposal has been given to the Department to conduct such an experiment 
over the next three j/ears at an annual cost of $500,000. 

Weaknesses 

No assurance of compatibility of data from school districts for state 
management decision making. 

Time involved in accomplishing an audit for a local system by Department 

staff. 

Expense to state in doing the data processing required for the analysis. 
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An option for project evaluation would be the consideration of a 
performance contract for project funds. The cost of evaluation would be a 
built-in factor of the contract and would be reported as performance delivered 
as stated in the grant. Ihis type of operation would establish an on-going 
evaluation procedure rather than an ex post facto analysis of the project. 

The Belmont Project is a systematic attempt to coordinate evaluation 
for Federal funds. The workload for the system at the state level has not been 
fully developed, but preliminary estimates indicate a full-time professional 
with seci^tarial assistance is a mlnlmiom requirement to coordinate the effort. 
This estimate does not reflect the cost of data processing or reporting results. 
It also omits the vital element of monitoring projects to provide services to 
the districts. 
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D. Syncreolc Model 

The Departmenc of Education is confronted with the problem of 
developing a system which provides the means for meeting its evaluation 
r' '^ponsibilities under State and Federal legislation. These responsibilities 
place the Department in the dual role of being held accountable, and at the 
same time, holding local school districts accountable to it. To meet these 
responsibilities, the model should be a unified system with a total capability 
for state assessment, district evaluation, and project evaluation. 

The purpose of a S5 'cretic model is to attempt to design a system v^ich 
preserves the strongest feati 'es of each program alternative idiile overcoming its 
major weakness. For example, the major question, "What are we purchasing for our 
educational expenditures in terms of performance? is not answered adequately In 
the program alternatives. 

The model r^ust also i^late to the projected future activities of the 
Department as can be determined from an assessment cf the current status. Tn 
this respect, three major events must be considered as influencing factors: 

1. The initiation of the s+*ate needs assessment project, 
"Our Schools"; 

2. The initiation of the preliminary study for a management 
information system;' 

3. Ihe responsibilities of the Department in the possible 
implementation of the Bateman State Aid legislation. 

The establishment of state goals and objectives for the educational 
system implies that indices of measurement will be developed which report current 
conditions. To develop state norms, the indices must be derived from standardized 
instruments \inlch are valf ' and reliable. The indices should be developed annually 
if needs assessment is to .e related to the Bateman Aid formula. 

With the utilization of standard instruments, comparable data could be 
available to the Department of Education and local school districts to construct 
pr^riles of districts with similar characteristics. The Department can identify 
problem areas from the profiles. This identification of a problem area is the 
first signal of the need for further problem analysis. While indices provide 
the evidence of vdiere and to what degree a problem exists, an analysis is 
necessary to detennine the cause and effect relations which exist and the 
solutions to be recommended. This analysis would be accomplished by on- 
site evaluation service teams from the Department. 

The Management Infoimation System (MIS) would provide the physical 
capability to store, process and retrieve information. The system, when 
developed, should prepare routine reports, have a rapid retrieval capability, 
and perform the statistical functions of problem analysis. A major output 
of the system would be the annual educational indices of state goals and 
objectives as measured by the needs assessment program. 

Solutions recommended in problem analysis could be in a research or 
project format. From the data bank of the MIS, the evaluation team would 
retrieve information on projects v*iich are related to the problem area. The 
recommendations would then be mere accurate predictions of expected outcomes 
from alternative solutions, in terms of reasonable expectations over given 
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time periods and at given expenditure levels. The selected alternative would be 
monitored as a project by the Department, and the outcomes would become input to 
the MIS data bank at the end of the project. 

An innovative research solution to the problem would require additional 
Department services. Stricter controls would be established, but the procedure 
for information flow to the data bank would be followed. 

The syncretic model is a two-step approach to evaluation with a 
continual process for information input, its major strength is the ability to 
identify problem areas in the educational system and to measure change over time. 
From Che identification of problem areas, the Department can structure priorities 
as the basis for concentrated Department on-site evaluation analysis services. 
The analysis would begin at the Department, where the r4IS would provide the 
evaluation team with historical data. 3 data could be in the form of school 
district profiles and compared over several years. The evaluation service team 
would then determine why differences exist and make recommendations which might 
be in the form of technical assistance to the local school districts. This 
process could be related, then, to the Federal and State grant-in-aid programs 
and projects. 

The Bateman State Aid legislation has implications which will require 
a realignment of the Department's current evaluation procedures. The Department's 
responsibility for the ce? tif ication of school districts into categories makes 
our present school approval program obsolete Certification requests on forms 
designed and supplied by the Department of Education could provide an information 
base for school district profiles as previously described. Certification also 
suggests the processes of review, approval and verification. Verification 
procedures could require an on-site team or an individual audit evaluation. 
The syncretic model has this feature. 

The syncretic model needs to be developed. At the present time, 
within the Department, an information base of process measures, i.e., tax base, 
expenditure level, organization of district, numbers of students by grades, 
numbers of professional staff, etc., is available to structure a state profile. 
However, without proc'uct or outcome measures, there is no means of relating 
this information to performance. 

The model is dependent upon a statement of performance goals and 
objectives. The initial stage, as we are presently proceeding in the "Our 
Schools Project*', is the development of the objectives. In the second stage 
assessment instruments should be constructed and distributed to the local 
school districts with directions for the adrjiinistratlon, scoring, recording 
and reporting. It is not necessary to wait until stage one is completed before 
stage two implemented, ihe implementation of stage two could begin in the next 
school year in the area of the basic skills at the elementary school level. 

It would be to the advantage of the Department of Education to begin 
the implementation of stage two. The information is basic to the model and 
benefits state assessment, school district evt^.luation and project evaluation. 
It will produce base-line data Imperative to planners at all levels. It is an 
information need idiich will be a major determinant in the development of the 
management information system. (The preliminary study for the design of the 
system will begin May 1, I970. The first phase is to determine +-,he data base 
required by the Department and to produce a ctaceptual model from this by 
August 1970). 
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The system approach of the syncretic model attempts to synchronize 
the separate evaluation functions being conducted within the Department at the 
present time. It shifts the emphasis from the present philosophy of evaluation 
as an approval concept to a future concept of evaluation as a problem analysis 
oriented services model. It provides services to Department staff and local 
school districts through its capability to generate relevant information in 
terms of cost-benefits and cost effectiveness to the achievement of predicted 
outcomes. It builds in a way to return information which assesses and verifies 
its own effectiveness. This loop provides ' ie basic information by idiich the 
Department's accountability to its constituents is evaluated. It measures the 
effectiveness of the decision of the past to produce change. 



VIII> BASIC CORE OF RECOMMENDATIONS 



Since September 1969, the State Board Sub- Committee on Evaluation and 
Testing has conducted a thorough examination of education evaluation practices 
in Nrw Jersey. The Sub- Committee 's work was based on the conviction that it is 
imperative to know the precise effects which education has upon the students of 
this State. Without such knowledge, our schools will lack the capacity for 
intelligent change, for sound program development, and for responding to the 
insistent call for accountability. 

The Sub- Commit tee ^s inquiries have revealed a substantial number of 
well-conceived evaluative activities. Nevertheless, the group found significant 
shortcomings which must be remedied if the Department of Education is to 
satisfactorily fulfill the service, re^julatory, and leadership roles it has 
undertaken. 

The following recommendations for action distill the Sub- Committee 's 
conclusions. They are divided into co.icerns i^iich require immediate action and 
those \inich must be considered in the years ahead. 



Recommendations for Short-Range Action 



A. School District Evaluation 

Ihe new evaluative and consultative obligations demanded by the Bateman 
legislation will make the existing school approval program obsolete. The 
Commissioner should ensure that Department personnel are reti'ained Tor their new 
functions and responsibilities. Local school districts should be involved in any 
new evaluation criteria developei. 

B. State-Wide Assessment 

The Sub-Committee noted with interest the increasing number of states 
adopting a state-wide assessment program. While the "Our Schools" project will 
include preliminary steps in this direction, further action seems required. 
The State Board should adopt a policy for state-wide testing of basic skills, 
and the Department should draw up plans for the e£.rly Implementation of this 
policy with involvement of the various publics concerned with education in 
the state. 

C. Project Evaluation 

The increasing number and complexity of Federal and State grants 
demand a greatly improved project evaluation system. All Department projects 
with large potential impact, as defined by the Jommissioner, should be evaluated 
once a year by an independent evaluation unit within the Department. Any 
re-approval action required by the State Board or the Commissioner should be 
supported by a statement evaluating the previous yearns activities. The 
Department's evaluation capacity should be strengthened accordingly, and 
additional necessary resources should be allocated for this purpose. Hopefully, 
a way will be found to evaluate projects within a more comprehensive framework 
O in the ent':\. local educational system. 
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Recommendations for Longer- Range Action 



A. Needs Assessment 

V/hlle basic skills testing Is an appropriate starting point, evaluation 
activities should contribute to the assessment of the specific education goals 
aiid objectives established through "Our Schools", the state-wide and local needs 
assessment program. Adequate means must be developed to measure progress In all 
major goal areas. 

B. Management Info mat Ion System 

Uie Department's Management Information System, presently In Phase I 
of developnent, should have the capacity to collect and store the data required 
for expanded evaluation activities of the Department, including school district 
evaluation, project evaluation, and state-wide assessment. 

C. Cost Analysis 

The recent efforts of the Department in the areas of ?PB and cost 
analysis should be continued and specifically related to longer-range 
evaluation activities. 

D. Instruction 

The Department should work with local school districts to develop 
suitable techniques for evaluating professional personnel. 

E. Department Task Analysis 

The Department should develop the capacity to periodically analyze 
and evaluate itself. 
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IX. ADDITIONAL QUESTIONS FOR CONSIDERATION 



A. What special studies are required to refine our knowledge of evaluation 
programs and techniques? 

B. Should we devise special mechanisms for the evaluation of teachers and 
administrators? 

C. How do you reduce the apprehension idiich students, parents, teachers, and 
administrators may feel vdien evaluative activities are intensified? 

D. Should we undertake a comprehensive evaluation of the internal functioning 
and capabilities of the Department of Education? 

E. Does the feasibility of state-wide testing require further examination? 

P. Should we recommend a halt to further development of the school approval 
system? 

G. As evaluation activities expand, can we anticipate any economies of scale? 
Or will administrative and organizational requirements produce cost 
diseconomies? 

H. Would it be preferable to locate the evaluative function outside of the 
Department? 

I. How do we cope with education's delayed payoff? 

J. Should the establishment of education goals for the State precede further 
development of any major evaluation program? 

K. Will the Department's forthcoming Management Information System adequately 
meet the needs of the syncretic model outlined in Part VII? 

L. What should be the role of extra-institutional evaluation agencies or 

groups; e.g.. Educational Testing Service, the College Entrance Examination 
Board, the National Assessment Project? 

M. How should we evaluate pre- school and continuing adult education programs? 

N. Should we weigh the desirability of evaluating non-public schools and pupils? 

0. What special problems ai^ raised by pupil mobility? 

P, What influence will the Federal Government have upon future evaluation 
activities in New Jersey? 

Q. Is it advisable to develop--and perhaps disseminate--a master plan for 
evaluation, covering objectives and activities for short-range, 
intermediate and long-range implementation? 
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APPENDIX A 

ANALYSIS OF DEPARTMENT OF EDUCATION SALARY COSTS 



1. Calculation of Work Year for 12-Month anployee . 
365 

-104 Saturdays and Sundays 
- 11 Holidays (average year) 

250 Basic work year, not Including sick leave and vacation, 
both of which vary according to type of personnel. 



Professional Personnel 



Unclassified 
250 

- 22 Vacation 
22^ 

- 5 Sick days* 

223 Potential work year 



Secretarial and 



Young 
250 

- 12 Vacation 

- 1^ Sick leave** 
223 



Classified 

250 

- 12 (used as average) 
23? 

- 5 Sick days* 

233 Potential work year 



Personnel 



Mature 

250 

- 12 Vacation 
23F 

- 5 Sick leave** 

233 



*Estlmated average usage. 

*It Is felt by the Personnel Office that the usage of sick leave varies 
with the maturity of the employee. 
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2. Calculation of Work Year for 10-Month Employee . 

365 

- 62 July and August 
303 

- 86 Saturdays and Sundays 
217 

- 4 Sick leave estimated 

213 

- 8 Holidays estimated 

205 

3. Calculation of Dally Salary at Selected Job Levels . 

The median salary for the particular range has been divided by the number 
of days In the potential work year. This figure Is entered In the salary 
column and represents a dally salary cost. In the next columrv a factor of 
2C$ for professional and for secretarial-clerical personnel has been 
added on. 



Job Classification Range No. 



Helping Teacher I 
County Supervisor 

of Child Study 
Supervisor I 
Director III 
Director II 
Director I 
Deputy Assistant 
Camnlssloner 



Clerk Stenographer 

Senior Clerk- 
Stc-nographer 

Principal Clerk- 
Stenographer 

Head Clerk 



28 

31 
32 
37 
39 
41 

42 



10 
14 

18 

22 



Potential Work 
Year (Days) 



205 

223 
223 
223 
223 
223 

223 



228* 

228* 

228* 
228* 



Salary 

$52.76 

56.14 
58.96 
75.25 
82.95 
91.46 

96.03 



$19.71 

23.96 

29.13 
36.19 



Salary Plus 
Fringe Benefits 

$ 63.31 

67.37 
70.75 
90.30 
99.54 
109.75 

115.24 



$ 23.06 
28.03 

34.08 
42.34 



*The days In the potential work year for secretarial-clerical personnel are the 
average of the "young" and "mature". 



Robert G. Bongart 
January, I970 



APPENDIX B 



OVERVIEW OF EDUCATIONAL TESTING SERVICE SURVEY 
OF STATE TESTING PROGRAMS 



Eight states indicated that they do not conduct a program. At least 
nine additional states provide only limited testing services. The 
foxlowing table portrays the number of states with one or more program:; 



The following 12 categories were developed to facilitate the 



compilation of pertinent data and to furnish a meaningful and consistent 
structure to each summary; 



Purposes and Objectives : state and federal legislation 
affecting the program; specific objectives such as 
guidance, placement, general assessment of academic 
progress, course evaluation, high school and college 
equivalency, college admissions, identification of 
special problems and talent 

Administration and Supervision ; the state governmental 
agency, educational institution, or other organiza- 
tion that conducts the program and provides services; 
the individuals that administer tests 

Grades : specific grades, grade levels, or ranges for v^ich 
the program is designed 

Tests ; titles, editions, levels, and forms of all instru- 
ments used; areas of testing such as intelligence, 
achievement batteries, achievement tests in individual 
subjects, multi-aptitude batteries, interest inventories, 
readiness tests, scholarship examinations; indication 
ofiiiether the tests are nationally produced, locally 
developed, or specially constructed by an outside agency 



Number of Programs 



Number of States 



1 
2 

3 
k 

5 
6 

7 

8 



2k 

13 
2 
1 
0 
1 
0 
1 



Norms : national or local norms, or both, and varieties of 
local nonns such as state, county, school district, and 
school 
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Administration Datei : exact or seasonal dates of test 
administration; Indication of vrtiether dates are 
left to the discretion of participating schools or 
school personnel 

Other Services : scoring, reporting, and special score 

Interpretation services; consultations with students 
and parents (guidance and counseling); professional 
workshops and In-servlce training; facilities for 
handicapped students 

Costs: student test fees, costs to participating schools, 
amount of reimbursement from state and federal govern- 
ments, special appropriations by state legislature 

Participation : voluntary or compulsory participation, pro- 
grams that are strongly recommended by the state, and 
those that are required for a specific purpose or In 
a certain context; types of exemptions such as mentally 
retarded students; differential participation by public, 
private, and parochial schools 

Number Tested : exact or approximate number of students 
tested annually and. If possible, the percentage of 
enrolled students that are tested 

Reference(s) : titles, authors, and publication dates of all 
materials and literature describing the program 

For Further Information : complete name, title, and address 
of the person from whom more detailed Information may 
by obtained 



A detailed analysis of the accumulated data on eight of these 
categories follows, with a delineation of national patterns and implica- 
tions. The Canal Zone, Guam, Puerto Rico, and the Virgin Islands are 
omitted In this analysis. 

Purposes and Objectives 
The specific purposes and objectives most frequently cited 
are listed In the following table: 
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Programs States 



Guidance (Vocational and 
Educational Planning) 


29 


22 


Evaluation of Instruction, 
OOUPS6S, ourrxcu-La, 
Programs 


25 


17 


Student Evaluation 
(Assessment of Academic 
Progress and Status) 




x^ 


Scholarships and Other 
Awards 


10 


5 


Problems and Talents 
^■Uiagnosbic lesuingy 


Q 

7 


7 
r 


College Admissions 


6 


6 


Placement and Grouping 
of Students 


5 


5 


High School and College 
Equivalency (Awarding of 
Diplomas and Credits by 
Examination) 


5 


3 


Others 


6 


3 


No Data Provided 


19 


17 



Administration and Supervision 



The number of programs conducted by the state, a college 
university, and by a combination of both are indicated below: 

Programs States 

State Governmental Agency 

Exclusively 50 26 

College or University 

Exclusively • 17 7 
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Programs States 

State Governmental Agency 
and College or University 

Xshared Responsibility) 7 9 



Grades 

Test s^^rvices are offered in one or more grade at each level by 
the following niimber of programs and states: 





Programs 


States 


K-3 


19 


u 


^-6 


28 


21 


7-9 


la. 


31 


10-12 


46 


32 


Adults 


5 


2 


Others 


6 


k 


No Data Provided 


11 


9 



Twenty states provide testing in at least one elementary grade 
(1-6) and one secondary grade (7-12). Thirteen states test students 
at all four levels in the elementary and secondary grades, and six 
additional states offer testing at three of these four levels. Seven 
states provide test services in every grade from 1 through 12. Persons 
classified as "Others" include high school graduates, college applicants, 
students in junior colleges, technical institutes, and regular four-year 
colleges, and members of the armed forces. 



Tests 

Tests are analyzed in terms of types, nationally standardized 
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instruments most frequently used, and sources of test contruction* 

Each principal type of test is administered in the following number 
of programs and states: 

Programs States 

Intelligence Tests 
(Scholastic or 
Academic Aptitude, 

Ability, Mental Maturity) 38 31 

Achievement Batteries 34- 27 

Achievement Tests in 

Individual Subjects 33 22 

Multi -Aptitude Batteries 20 17 

Interest Inventories 7 5 

Readiness Tests 7 5 

Others lU 8 

No Data Provided 9 8 

Types of instruments may also be compared on the basis of total 

occurrence (frequency) and number of different tests. Only nationally 

standarized tests are included under "Different Instrimients." 

Different 

Frequency Instruments 

Achievement Tests in 

Individual Subjects 210 H 

Achievement Batteries 80 21 

Intelligence Tests 65 13 

Multi -Aptitude Batteries ''25 5 

Interest Inventories 11 U 

Readiness Tests 7 1 



Differe nt 
Frequency Instruments 

Others 24. 8 

Intelligence and achievement testing of some kind are offered in 

35 programs and 30 states. Meas^ores of achievement (batteries and/or 

tests in individual subjects) are provided in 56 programs and 36 states 

Eleven programs and 13 states offer both types of achievement testing. 

Nationally standardized tests most often mentioned are: 

PrograiTxo States 

Lorge-Thomdike Intelligence 
Tests 12 11 

Differential Aptitude Tests 12 10 

School and College Ability 

Tests 11 10 

California Short-Form Test 
of Mental Maturity 10 10 

Iowa Tests of Educational 
Development 10 10 

Stanford Achievement Test 10 9 

Sequential Tests of 

Educational Progress 9 8 

California Achievement Tests 7 7 

Iowa Tests of Basic Skills 7 7 

Me'^ropolitan Achievement 
Tests 6 5 

Otis Quick-Scoring Mental 
Ability Tests 5 5 

SRA Achievemen"c Series 5 5 
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The most frequently used nationally standardized test or battery 
of each type is listed below: 

Achievement Battery - Iowa Tests of Educat^ional 

Development 

Achievement Test in 
Individual Subject « Cooperative English Tests 

Intelligence Test - Lorge-Thorndike Intelligence 

Tests 

Interest Inventory - Strong Vocational Interest 

Blank (for Men and Women) 

Multi-Aptitude 

Battery - Differential Aptitude Tests 

Readiness Test - Metropolitan Readiness Tests 

Another significant question is whether a program makes use of 
nationally standardized tests directly available from test publishers, 
contructs its own instruments to meet individual needs, or employs the 
services of an outside agency to develop tests designer" specifically for 
the program. The prevalence of these three types of arrangements, as 
well as certain combinations, may be seen in the following table: 

Programs States 
Nationally Standardize I 54. 33 

Locally Constructed (By 
State, School District, 

or School) 17 q 

Developed by Outside 

Agency 7 ^ 

Nationally Standardized 
and Locally Constructed 7 6 

Nationally Standardized 
and Developed by Outside 

Agency 1 3 
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Programs States 

Locally Constructed and 
Developed by Outside 

Agency 0 1 

Nationally Standardized, 
Locally Constructed, and 
Developed by Outside 

Agency (All Three) 0 1 

No Data Provided 3 3 



The number of tests of each variety, in terms of frequency and 
different instruments, appears below: 

Different - 
Frequency Inscruments 

Nationally Standardized 253 66 

Locally Constructed 15^ 151 

Developed by Outside 

Agency 17 15 

Total Number of Tests ^26 232 

Norms 

Twenty-five programs and 18 states offer local norms for nationally 
standardized tests. An additional 21 programs and nine states utilize 
locally constructed tests or instruments developed by an outside agency. 
Presumably these 21 programs and nine states provide local norms. 
Proceeding on this assiimption, one might logically conclude that local 
norms are made available in approximately 4-6 programs and 27 states. 
No data concerning norms are provided i^T 12 programs distributed among 
11 states. 
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Administration Dates 
The period of test administration most frequently mentioned is the 
fall (September throiigh November), with October the month most often 
listed. Tests are administered strictly at the discretion or convenience 
of participating schools in 31 programs representing 22 states. A 
classification by month yields the following figures: 



Programs States 

September 11 n 

October 17 16 

November 12 9 

December 4 3 

January 5 5 

February 5 5 

March 7 6 

April 5 5 

May 4 3 

June 2 2 

July 1 1 

August 1 1 

Test administration times without specific month designations are 
indicated below: 

Programs States 

Fall 2 2 

Spring 3 3 

Beginning of School Year 3 2 

End of School Year 2 2 
Periodically or Throughout 

the Year 4 3 

No Data Provided 5 4 



Other Services 

Services, activities, and materials connected in some way with a 
testing program are often omitted in summaries prepared by states. 
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The following figures are the result of tallying the number 
of programs and states in which a certain service was definitely- 
mentioned. They should therefore be regarded as highly tentative. 

Programs States 

Reporting A3 25 

Scoring ^2 25 

Scoring and Reporting 4.0 2U 

Special Score Interpretation 
Materials and Services (Such 
as Item Analysis) 32 19 

Professional Workshops and 
In-Service Training 14, 11 

Guidance and Counseling 
(Consultations with Students 
and Parents) 12 8 

Other Services 14, 11 

No Data Provided 12 8 

Included among "Other Services" are research programs, facilities 
for the handic'jtpped (such as Braille tests), biographical forms, and 
special provisions for make-up testing or retesting. 

Participation 

With regard to participation, it is difficult to establish a 
distinct voluntary-compulsory dichotomy. A testing program may be 
completely voluntary for private schools and certain age groups, 
compulsory fc^ public schools and specific grade levels, and strongly 
recommended for other institutions. A program may be generally voluntary 
(such as scholarship examinations) , but an individual mi:ist participate 
O if he wishes to earn a particular type of scholarship. Equivalency 

e:-aminations are commonly taken by students and veterans seeking to 
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obtain high school or college credits or high school diplomas. The 
decision to participate is entirely theirs, but the test or battery 
given for this purpose may be the only one acdepted by the colleges 
of their choice. 

In this overview, most programs that are not strictly com- 
pulsory will be considered voluntary. This category embodies all 
scholarship programs, high school and college equivalency programs, 
as well as programs authorized by Title V-A of the National Defense 
Education Act. Any testing program required of public school students 
in selected grades is classified as compulsory, although it may be 
optional for nonpublic schools. Certain exemptions, especially for 
the mentally retarded, are expected in virtually all programs. Within 
the framework of this rather tenuous classification scheme, the follow- 
ing table was constructed; 

Programs States 
Voluntary 64 54 

Compulsory 8 5 

Combination of Voluntary 

and Compulsory Programs — 1 

No Data Provided 2 2 

The appearance of an asterisk (*) in any state summary denotes 
that no data were provided for that category. 
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